Convex Optimization for Parameter Synthesis in MDPs

Probabilistic model-checking aims to prove whether a Markov decision process (MDP) satisfies temporal logic specification. The underlying methods rely on an often unrealistic assumption that the MDP is precisely known. Consequently, parametric MDPs (pMDPs) extend with transition probabilities are functions over unspecified parameters. parameter synthesis problem compute instantiation of these parameters such resulting We formulate as quadratically constrained quadratic program, which nonconvex and NP-hard solve in general. develop two approaches iteratively obtain locally optimal solutions. first approach exploits so-called convex–concave procedure (CCP), second utilizes sequential convex programming (SCP) method. techniques improve runtime scalability by multiple orders magnitude compared black-box CCP SCP merging ideas from optimization probabilistic model-checking. demonstrate satellite collision avoidance hundreds thousands states tens their wide range commonly used benchmarks.